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QUEUEING SYSTEMS WITH MANY SERVERS: NULL 
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A queueing model has J > 2 heterogeneous service stations, each 
consisting of many independent servers with identical capabilities. 
Customers of I > 2 classes can be served at these stations at different 
rates, that depend on both the class and the station. A system ad- 
ministrator dynamically controls scheduling and routing. We study 
this model in the central limit theorem (or heavy traffic) regime pro- 
posed by Halfin and Whitt. We derive a diffusion model on R 1 with 
a singular control term that describes the scaling limit of the queue- 
ing model. The singular term may be used to constrain the diffusion 
to lie in certain subsets of R 1 at all times t > 0. We say that the 
diffusion is null- controllable if it can be constrained to X_, the mini- 
mal closed subset of R 7 containing all states of the prelimit queueing 
model for which all queues are empty. We give sufficient conditions 
for null controllability of the diffusion. Under these conditions we 
also show that an analogous, asymptotic result holds for the queue- 
ing model, by constructing control policies under which, for any given 
< e < T < co, all queues in the system are kept empty on the time 
interval [s,T], with probability approaching one. This introduces a 
new, unusual heavy traffic "behavior": On one hand, the system is 
critically loaded, in the sense that an increase in any of the external 
arrival rates at the "fluid level" results with an overloaded system. 
On the other hand, as far as queue lengths are concerned, the system 
behaves as if it is underloaded. 

1. Introduction. We consider a multiclass queueing model with hetero- 
geneous service stations, each consisting of many independent servers with 
identical capabilities. The servers offer service to different classes of cus- 
tomers at rates that may depend on both the station and the class. A sys- 
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tem administrator dynamically controls all scheduling and routing in the 
system. The model is considered in the heavy traffic parametric regime, 
first proposed by Halfin and Whitt [7], in which the number of servers at 
each station and the arrival rates grow without bound, while keeping, in 
an appropriate sense, a critically balanced system. Both the model and the 
parametric regime have recently received much attention, especially in rela- 
tion to large telephone call centers (see [6] and references therein). 

When studying queueing models in heavy traffic, one considers a sequence 
of models parametrized by n € N that, under a law of large numbers (LLN) 
limit, give rise to a fluid model which is critically loaded. Typically, an 
attempt is then made to prove that appropriately scaled fluctuations of the 
queueing model about the fluid model converge to a diffusion. If, as in the 
current setting, a control problem is associated to the queueing model, then 
a similar approach gives rise to a controlled diffusion model. In this case, a 
natural goal is to prove that, given a cost criterion, the (suitably scaled) value 
of a control problem for the queueing model converges to one for the diffusion 
model. Moreover, it is often the case that solving the diffusion model for 
optimal controls helps understand how to construct control schemes for the 
queueing model that are asymptotically optimal. A similar approach is taken 
in the current paper. However, rather than a cost criterion, our formulation 
will be concerned with a certain property observed for the diffusion model 
and shown to be inherited, in an asymptotic sense, by the queueing model. 
The property, which is unusual in heavy traffic formulations, will have to do 
with the ability to maintain empty queues. 

We let / denote the number of customer classes, and let X n (t), t>0, 
denote an /-dimensional process for which the ith component Xf{t) repre- 
sents the total number of class- i customers in the system at time t, in the 
nth system. The fluctuations alluded to above are denoted by X n (t), i > 0. 
These fluctuations are scaled in such a way that X n gives rise, as weak lim- 
its are taken formally, to a (controlled) diffusion process denoted by X(t), 
t > 0. Moreover, one has that 



r(t)ex_ := 




holds if and only if the total number of customers in the nth system at time 
t is less than or equal to the total number of servers. Ideally, if one could 
freely rearrange customers in the system without any constraints, it follows 
that one could maintain empty queues whenever X n {t) € X_. We will thus 
refer to X_ as the null domain. Although the queueing model considered 
in this paper is subject to additional constraints (e.g., a station may offer 
service to only some of the classes), the null domain will play the same role 
in the asymptotic regime under consideration. 
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An important feature of the controlled diffusion model derived in this 
paper is that the stochastic differential equation describing it has a singular 
control term, that is, a control process with sample paths that are locally of 
bounded variation, the increments of which take values in a fixed cone C of 
M. 1 . The singular term may be used to constrain the diffusion to lie in certain 
subsets of at all times t > 0. We say that the diffusion is null- controllable 



where X° denotes the interior of X_, because, under this condition, the 
diffusion can be constrained to X_. Condition (1) will be given in explicit 
form in terms of the model parameters [see (33)]. Our main result shows 
that under (1) one can construct control policies for the queueing model 
in such a way that, for every given < e < T < oo, all queues are kept 
empty on the time interval [e,T] with probability approaching 1, as n — > oo. 
We will refer to such behavior as asymptotic null-controllability. We will, in 
fact, consider two versions of the problem: One, referred to as preemptive 
scheduling, in which service to a customer can be interrupted and resumed 
at a later time (possibly in a different station). The other, referred to as non- 
preemptive scheduling, where service to customers may not be interrupted 
before service is completed. The treatment of the nonpreemptive case is 
more complicated than that of the preemptive case. Thus, to keep the ex- 
position simple, we have limited ourselves in the nonpreemptive case to the 
simplest possible network structure where null-controllability can show up: 
two customer classes and two service stations. 

Our results on asymptotic null-controllability can be regarded as the 
demonstration of a new, unusual heavy traffic behavior. On one hand, the 
system is critically loaded. Indeed, as intuitively expected (and precisely 
stated in Proposition 2.1), an increase in any of the external arrival rates 
at the fluid level results with an overloaded system, in the sense that large 
queues inevitably build up. On the other hand, the system behaves as if 
it is underloaded as far as the capability of maintaining empty queues is 
concerned. 

Singular control arises in connection with queueing systems in heavy traf- 
fic in many references. The singular term is often associated with positivity 
or finite buffer constraints for the queue length process (see [9], Chapter 8, 
for discussion and further references), with admission control (ibid.) or with 
constraints on the so-called workload process to lie in a given cone [5]. The 
source for the singular term in the current setting is, however, quite different, 
and it has to do with the fact that a many-server limit is taken. To explain 
this point, consider a network in which customers of classes, say, 1 and 2, 
can be served at both station A and station B (these two classes and two 
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stations could be a subset of a larger setup). Assume that the network op- 
erates under preemptive scheduling. Suppose that at a certain moment one 
selects, say, r class-1 customers that are in service at station A and r class-2 
customers in station B and considers the option of interchanging their posi- 
tion, so that the r class-1 (class-2) customers that were selected are moved 
to station B (resp., A). Since the service rates may depend on both the class 
and the station, the average rate at which components 1 and 2 of X n change 
at that moment may be different depending on whether such an interchange 
takes place or not. If the interchange is performed instantaneously, the rates 
alluded to above will change abruptly. Moreover, since in both stations the 
number of servers is assumed to be large, this effect can be amplified by 
letting r be large. This, as scaling limit is taken, results with a control term 
that can have arbitrarily large increments over a given time interval. An ap- 
propriate formulation for such a controlled diffusion model will thus have a 
singular control term. Next, let Q denote the graph with classes and stations 
as vertices, and with (i, j) pairs as edges if and only if station j can serve 
class z. It is instructive to note that our explanation above relies on a certain 
property of the graph Q, namely, that it contains a cycle with vertices 1,2, 
A and B. Indeed, this is just another way of saying that customers of each 
of the two classes can be served in both stations, as we have assumed. A 
heuristic similar to the one described above will lead to a singular term in 
the diffusion model whenever Q contains cycles, whether with four vertices 
or more. Thus, in general, cycles contained in Q will play an important role 
in the singular control formulation of the diffusion model. 

Note that the phenomenon described above is indeed a result of the many 
server setting, because it must be possible to ocasionally let the number r 
referred to above take large values, and r is clearly limited by the number of 
servers. In particular, this phenomenon is not seen in what is sometimes re- 
ferred to as "conventional" heavy traffic, where diffusion limits are obtained 
for systems with a fixed number of servers (and large service rates; cf. [11]). 

Recall that the so-called fluid model describes the LLN limit of basic 
quantities of the queueing model. One ingredient of the fluid model is a 
(deterministic, constant) matrix denoted by £*. The entry represents 
the (large n limit) fraction of the number of servers in station j that serve 
class-i customers. One refers to pairs that are edges of Q as activities, 
and to activities (i,j) for which £*• > as basic activities. In particular, 
the number of servers that are engaged in nonbasic activities = 0) is of 
order o(n) (where the total number of servers is proportional to n), in a 
sense that can be made precise (see, e.g., Lemma 4.1). The policy that we 
shall propose for the preemptive case will basically mimic the construction 
of a constrained diffusion. Namely, a special rearrangement of customers in 
the service stations will take place whenever the process X n reaches close to 
the boundary of X_ (from the inside). In this rearrangement, the number 
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of servers allocated to work in a certain nonbasic activity will be much 
larger than the typical fluctuations of the process X n [but still o(n)]. This 
rearrangement will have an effect on the dynamics of X n that is a reminiscent 
of that of a Skorohod mechanism [10] on a constrained diffusion. Namely, 
it will constrain X n to X_ by making the term that, in the limit, shows 
up as singular control, large. Because of (1), this can be performed in such 
a way that the term points into the domain. Although this question is not 
directly addressed in this paper, it is expected, in fact, that the restriction 
to the time interval [e,T] of the processes X n converge to a constrained 
diffusion as n — > oo. The picture is quite different in the nonpreemptive 
setting. The arrangement of customers can only be controlled indirectly (via 
routing decisions), and the constraining mechanism of the diffusion cannot 
be faithfully mimicked. A convergence result as above is not to be expected; 
in fact, the processes X n that we construct in this case are not even tight. 
The main idea behind the policy proposed in this case is to assure that a 
relatively large portion of the servers are engaged in the nonbasic activity 
at all times, rather than at times when the boundary is reached as in the 
preemptive case. 

We reiterate that it is the presence of cycles in Q that induces a singular 
control term in the diffusion. A model similar to the current one is studied 
in [1] and [2] under structural assumptions that are complementary to those 
of this paper, in the sense that the graph Q is assumed there to be a tree. 
Indeed, in these references the diffusion has no singular control term, and 
a phenomenon as described in this paper does not occur. Finally, we would 
like to mention that one can also consider a setting in which the diffusion 
has a singular term as in the current paper, but null controllability does not 
hold, and approach the model from a control theoretic viewpoint so as to 
minimize costs of interest. This will be subject for future study. 

The organization of the paper is as follows. Some notation is introduced 
at the end on this section. In Section 2 we first introduce the model and de- 
scribe how its parameters are rescaled. We then present the main step toward 
the derivation of the diffusion model in Theorem 2.1, where a representation 
of the prelimit queueing model is provided. The diffusion model, obtained 
from this representation under the scaling limit, is stated in equations (30)- 
(32). We then state that, under (1), the diffusion can be constrained to 
X_ (Theorem 2.2), and provide the main results regarding asymptotic null- 
controllability in the preemptive case (Theorem 2.3) and in the nonpreemp- 
tive case (Theorem 2.4). At the end of Section 2 we give numerical examples, 
and demonstrate that asymptotic null-controllability cannot be expected in 
overloaded models (Proposition 2.1). Section 3 contains the proofs of The- 
orems 2.2 and 2.3. The proof of Theorem 2.4 appears in Section 4. The 
Appendix contains the proofs of Theorem 2.1, Proposition 2.1 and some 
auxiliary results. 
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Fig. 1. (a.) A network with 2 classes and 3 stations, (h) Corresponding graph Q with 
basic and nonbasic activities (solid and, resp., dashed lines). The subgraph Qb of Q is a 
tree. 

Notation. For x € M 7 , let ||x|| = J2l=i \ x i\- For two column vectors v,u 
of the same dimension v ■ u denotes their scalar product. The symbols 
denote the unit coordinate vectors and e = (1, . . . , 1)'. The dimension of e 
may change from one expression to another. For E a metric space, we denote 
by ~B(E) the space of all cadlag functions (i.e., right continuous and having 
left limits) from R + to E. We endow B(i?) with the usual Skorohod topology 
(cf. [4]). If X n , n £ N and X are processes with sample paths in O(E), we 
write X n =4> X to denote weak convergence of the measures induced by X n 
[on B(i2)] to the measure induced by X. For a collection A of random 
variables, ct{.4,} denotes the sigma-field generated by this collection. If X is 
an Revalued process (or function from R + to IR 7 ), \\X\\f = sup < u <t ||X(it)||, 
and if X takes real values, \X\* = sup 0<u<t 

2. Setting and main results. 

2.1. Queueing model. A precise description of the queueing model is 
as follows. A complete probability space (£l,J-, P) is given, supporting all 
stochastic processes defined below. The processes will all be constructed in 
such a way that they have cadlag sample paths with probability 1. Expecta- 
tion with respect to P is denoted by E. The queueing model is parametrized 
by n £ N. It has I >2 customer classes and J > 2 service stations [see Fig- 
ure 1(a)]. Station j has iVj 1 identical servers working independently. The 
classes are labeled as 1, . . . ,1 and the stations as I + 1, ...,/ + J: 

J={1,. ..,/}, J = {I + 1,...,I + J}. 

Arrivals are modeled as renewal processes with finite second moment for 
the interarrival time. More precisely, we are given parameters A™ > 0, 
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n E N, and independent sequences of strictly positive i.i.d. random vari- 
ables {Ui(k),k E N}, i £ I, with mean EUi(l) = 1 and variance = 

Var(£/j(l)) E [0, oo). With J2i = 0, the number of class- i arrivals up to time 
t for the nth system is given by 

^(t)=sup{z>0:£^<ij, t>0. 

For i El,j E J and n £ N, we are given parameters u^- > 0, representing 
the service rate of a class- z customer by a server of station j. If = 0, we 
say that class- z customers cannot be served at station j. Consider the graph 
Q having a vertex set X U J and an edge set 

S = {(i,j) elx J-.fj.Vj >0}. 

We assume that £ does not depend on n. We denote i ~ j if (i, j) € £. A 
class-station pair is said to be an activity if i^j, or, equivalently, if 
class-z customers can be served at station j. For every (i,j) El X J, we 
denote by ^fj(t) the number of class- i customers being served in station j 
at time t. By definition, 

(2) for^i- 

Service times are modeled as independent exponential random variables. To 
this end, let S"™-, (i, j) El x J be Poisson processes with rate (where a 
Poisson process of zero rate is the zero process), mutually independent and 
independent of the arrival processes. Note that the time up to t devoted to a 
class- i customer by a server, summed over all servers of station j, is given as 
Jq ^fj(s) ds. The number of service completions of class-i customers by all 
servers of station j up to time t is, by assumption, given by «S^(/o ^2j(s) ds). 
The precise description of the processes \I/ n = (^fj,i El,jE J) is not given 
at this point. We do emphasize, however, that they will be constructed in 
such a way that future service completion times are independent of the cur- 
rent state, which results with independent exponential service times (cf. [2]). 
We note in passing that whereas renewal arrivals are quite natural in the 
Halfin-Whitt setting, assumptions on service times that go beyond expo- 
nential (not to be dealt with in this paper) lead to far more complicated 
diffusion models [12]. 

The processes A n and S n will be referred to as the primitive processes. 

Denoting by Xf(t) the number of class- i customers in the system (mean- 
ing in the queue or being served) at time t, and setting X®' n = Xf (0), it is 
clear from the above that 

(3) Xf{t) = X°> n + A?(t) - E S 5 ( f *U S ) ds ) > i El,t>0. 
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For simplicity, the initial conditions X i ' n are assumed to be deterministic. 
Finally, we introduce the processes Y™(t), representing the number of class-i 
customers in the queue (not being served) at time t, and Z™{t), representing 
the number of servers at station j that are idle at time t. Clearly, we have 
the following relations: 

(4) + i€T ' 

(5) 2%(t) + ^J- 
Also, the following holds by definition: 

(6) ^ n (t)>0, Zf(t)>0, *?■(*)><), i€l,j€j,t>0. 

We will write X n for the vector (Xf , i £ 1) and similarly Y n = (Y™, i&T), 

z n = (Zf,jej). 

2.2. Control and rescaling. Equations (2)-(6) indicate some properties of 
the processes involved, but they do not characterize these processes, because 
the control processes \l/ n have not yet been described. This is the purpose 
of the following definitions. 

Preemptive scheduling. We will regard scheduling as preemptive if service 
to a customer can be stopped and resumed at a later time, possibly in a 
different station. Formally, such a scheduling is a scheme according to which 
one selects ^ n (t) at every t. In this paper we will be concerned only with 
scheduling of feedback form, in the sense that the selection of ^ n (t) depends 
only on X n (t), for every t. The precise definition is as follows. 

Definition 2.1. Let n be given. We say that a map / n :Z^_ — > Z+ J is 
a preemptive resume scheduling control policy (P-SCP) and X n is the con- 
trolled process corresponding to f n , initial data X 0,n and primitive processes 
A n and 5 n , if * n (i) = f n {X n {t)) and equations (2)-(6) hold. 

Nonpreemptive scheduling. By this, we mean that it is impossible to 
interrupt service to customers. Thus, the quantities ^ffj cannot be directly 
controlled, but they are affected by the routing decisions according to the 
following equation: 

(7) = ¥« (0) + fl»(t) - S% QT cto) • 

Above, for each Gl x J~, is a nondecreasing Z + -valued process 
starting from zero, that increases by k each time k class-* customers are 
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routed to station j. Of course, = for i ^ j. In nonpreemptive scheduling, 
a control policy is a scheme for selecting BfAt) for every t. In this paper 
we will need a randomized formulation, in which the scheme according to 
which Bfj are determined may depend on an auxiliary stochastic process. In 
addition, in our formulation we will only need the routing decisions to take 
place at the times when arrivals occur. To this end, for i G T and n G N, we 
let T^' 1 denote the time of the kth jump of the process Af, that is, the time of 
the kth class-i arrival. Finally, it will be assumed that all customers in service 
at time zero begin their service at that time, and the initial arrangement of 
these customers in the stations, that is, ^ (O), is determined by the policy. 
We write Z n (t) = (X n (t),Y n (t), Z n (t), ^ n (t)). 

Definition 2.2. Let n be given. We say that a triplet (^°' n , F n , R n ) is a 
non-preemptive scheduling control policy (N-SCP) and X n is the controlled 
process corresponding to (ty 0,n ,F n ,R n ), initial data X°' n and primitive pro- 
cesses A n , S n if the following hold: 

(i) ^°' n G Z J + J is the initial condition for that is, * n (0) = ^°' n . In 
particular, it satisfies *°j n = for i + j, *£j ^ < X°' n and £i *°j n < Nf. 

(ii) R n is a collection {R n '\ i G J}, where, for each i, R n ' { = {R^ : k G N} 
is a sequence of M-valued independent random variables. The sequences R n ' z 
are mutually independent and independent of the primitive processes. 

(iii) F n is a collection {F^, G T x J} of measurable maps FJ 1 - : M + x 
Zj.xZ|xZ{xZ 7 + J xK^ Z+ and for each el x J, B% is given in 
the form 

E F^\E n (T^-),R n ^), t>0. 

(iv) Equations (2)-(7) hold. 

Remark 1 . The restriction to policies in which decisions take place only 
when arrivals occur may not appear to be natural, and one could consider 
extensions of this definition, say, by allowing decisions to take place upon 
arrivals or service completions, or even continuously in time. Note, however, 
that this restriction does not affect our result (i.e., Theorem 2.4) that is 
concerned with the existence (and construction) of N-SCP with a certain 
property: Clearly, the existence of such policies under the preceding defini- 
tion implies the existence of policies under any extention of it. 

Remark 2. Existence and uniqueness of the processes X n and ^ n 
(given the primitive processes) is easily obtained by induction on the jump 
times of the primitive processes. In addition, one can argue that the future 
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service completion times are independent of all that has occured up to the 
current time. For a precise statement and an argument to this effect in a 
more restriced setup, see Proposition 1 of [3]. This can be adapted to the 
current setup, but we have omitted the details. 

Fluid scaling. We assume that the parameters of the processes involved 
are scaled in the following way. There are constants \,Vj E (0,oo), i El, 
and £ (0, oo), (i,j) E £ such that 

(8) rT^-Ai, n^N^Uj. 

We also define fj^j = for i ^ j. Note that this is consistent with (8) because 
Hij = for i j . Let 

(9) fly = Ujfiij, El x J, 
and consider the following linear program: 

Minimize p E R+ subject to 

(10) 

Yl >''J^'j = - P> - °' iEl, j Ej. 

Throughout, we assume that the fluid model is critically loaded. More pre- 
cisely, we will assume that the heavy traffic condition [8] holds: There exists 
a unique optimal solution to the linear program (10) and, moreover, 

YUexStj = 1 f° r all i 6 J7 (and, consequently, p* = 1). We shall keep the 
notation throughout the paper. We also let 

(11) Y^i]j"j- 1>i3=&3"j, iEljEj, 

j 

and refer to the quantities £*,^*,x* as the static fluid model, or just fluid 
model for short (see in [2] what these quantities intuitively represent). 

Following [8], an activity (i,j) E £ is said to be basic if > 0. Define 
the graph of basic activities Qt, to be the subgraph of Q having X U J as a 
vertex set, and the collection £& of basic activities as an edge set. We will 
also denote the set of nonbasic activities as £ n f> = £ \ £ b [see Figure 1(b)]. 

Like in some other papers that study a similar fluid model in heavy traffic 
(e.g., [8]), we will have one more assumption in this paper about the fluid 
model, namely, that the complete resource pooling condition holds. This 
condition expresses, in a sense, a strong mode of cooperation between the 
service stations. More precisely, one of the equivalent formulations of this 
condition (see [8]) is that all vertices in J communicate via edges in Q^. It 
was shown by Williams [13] that this condition is equivalent to the condition 
that the basic activities form a tree. Thus, we assume throughout: 



(12) 



The graph Qf, is a tree. 
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We now introduce a rescaled version of the processes describing the queueing 
model: 

X?(t)=n- 1 X?(t), 
Y?(t)=n- l Y i n (t), 
Zj(t)=n~ l Z?(t), 

Denote X n = {Xf, i El), and use a similar notation for Y n , Z n and ty n . We 
will sometimes consider X n , Y n and Z n as column vector- valued processes. 
Heuristically, one expects that (X n ,Y n ,Z n ,^ n ) =3- (x* ,0,0,ip*), and this is 
indeed the case under appropriate conditions [see, e.g., equation (57) and 
Lemma 4.1 for such statements in the preemptive and, resp., nonpreemp- 
tive case]. For this reason, these processes are referred to as the fluid-level 
rescaled processes. 

Diffusion scaling. We further assume that there are constants \i,fiij E K, 
i G 2", j G J, such that 

(13) A? := nV2 (n -l A n _ A .) ^ ^ *n . = n l/2^n _ ^ ^ ^ 

(14) N™: = n l/2 {n- l N™ 

We introduce a centered, rescaled version of the primitive processes: 

(15) A^t) = n-V\A?(t) - A?t), S%(t) = n-y\S%{nt) - n^t). 

Similarly, the processes representing the queueing model are centered about 
the fluid model quantities and rescaled: 

(16) X?(t)=n- 1 /\ X ?(t)-nx*), 

(17) Y?(t) = n- l ' 2 Y l n it), Z r ;{t)=n- l ' 2 Z^{t), 

(18) *?.(t)=n- 1 /2(*»(t)-^.n). 

The processes denoted with hats will be referred to as diffusion-level rescaled 
processes. Similarly to the fluid- level processes, define X n = {Xf ,i € I), with 
an analogous definition for Y n , Z n and *ff n , and consider X n , Y n and Z n 
as column vector-valued processes. 

Scaling of initial conditions. It is assumed that there are constants Xj, 
i G I, such that the initial conditions satisfy 

(19) X;> n :=X?(0)^Xi. 
Throughout, x= (xi,i GX). 
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2.3. Main results. Our first result expresses a relation directly between 
the diffusion-level processes. Although its proof requires only some elemen- 
tary manipulations of the relations (2)-(6) and (16)— (18), it has an im- 
portant role in this paper as the basis for deriving the diffusion model. In 
particular, it will make clear how the singular control formulation arises. To 
present it, we need some notation. 

Denote by C the set of all cycles that are subgraphs of Q, for which 
exactly one edge is a nonbasic activity. We call an element c G C a simple 
cycle [see Figure 2(a)]. Lemma A.l in the Appendix shows, using (12), that 
every nonbasic activity belongs to a simple cycle (as an edge). Consequently, 
there is a one-to-one correspondence between £ n & and C, which we denote 
by a. More precisely, 

a(i,j) = c whenever G £ n b and c is the simple cycle through (i,j). 



With an abuse of notation, we will write G c when we mean that an 
(not necessarily nonbasic) activity (i,j) G £ belongs to the edge set of the 
graph c. 

Next, we associate directions with the edges of simple cycles. Let c be a 
simple cycle with vertices io, jo,h, ji, ■ ■ ■ ^kiJk-i where for < I < k, i[ G T 
and ji G J, and edges (i ,jo) G £nb and (jo,k), (ik,jk), (jfc,«o) € E b . The 
direction that we associate with the nonbasic element (io,Jo) is *o ~~ * jo 
(in words, from io to jo). The direction of the other edges, when consid- 
ered as edges of c, is consistent with that of the nonbasic element, namely, 
io jo h ji —* jk —* io- Note that an edge (corresponding to a 

basic activity) may have different directions when considered as an edge of 
different simple cycles. We signify the directions along the simple cycles by 
s(c,i,j), defined, for all c G C and (i,j) G c, as 



(20) 




1 



1, if considered as an edge of c, is directed from i to j, 

if considered as an edge of c, is directed from j to i. 



We will denote 



(21) 



n 



s(c,i,j)ii, 



n 



iel. 



m^ = {ml c ,iel). 



Next, consider the system of equations in ip: 



(22) 



' ^2 = ai 



iel 



> = o, 
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It is known that (22) has a unique solution tp for every a, b satisfying J2i a i = 
J2jbj (see [1], Proposition A.2). With 



(23) 



D G = l(a,b)eR I xR J :Y,a i =Y; b j\> 



denote by G : Dq — ► K J the solution map, namely, 
(24) ^ - = G -(a,&), (i,j)e£. 

The function G is linear and so Lipschitz (a fact that will be used in the 
sequel). Denote also 

Hf(a,6) = -J2^ijGij(a,b), i e I, a 6 R 1 , b G R J , 

(25) 

H n = (H?,iel). 
Let # = A? - E iG j and set 

(26) Wm = A?(t) - J2 Sij *£W ds) + 

Finally, let the quantities {^^ j, as ranges over the nonbasic activities, 
be labeled by simple cycles, namely, define for every cGC, ^ c = VPy, where 
(i,j) = <t~ 1 (c). Let a diffusion-level version of these processes be defined as 

Theorem 2.1. LetX n ,Y n ,Z n ,^ n satisfy (2)-(6) and let X n ,Y n , Z n ,# r 
be defined by ( Then the following relations hold for all t > 0: 

X n (t) = X°' n + W n (t) + f H n {X n (s) - Y n (s) , N n - Z n (s) ) ds 

(27) 



" JO 

y™(t)>0, iGl, Z?(t)>0,j€j, 



(28) 



(29) 
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See the Appendix for a proof. We now explain how a diffusion model 
is derived from the above result. First, note that the limit, as n — > oo, in 
the definition (21) of m" exists, and is equal to the expression obtained 
from (21) by replacing fji^ by fiij. We denote the limit by m c . The vectors 
m c will be referred to as directions of control because of the role they will 
play in the singular control term. In a similar fashion, we denote by H the 
limit of H n as n — > oo, or, equivalently, as the expression obtained in (25) 
by replacing /ifj by fUj. Next, we take a formal limit in (26). We let £ = 

(£i,i 6l), where 4 = linv^oo^ 1 = Aj — J2jeJ frij^ij an d let W denote a 
Brownian motion for which the mean and the covariance matrix of W(l) 
are given by £ and, respectively, £ := diag(AjCjjj + Aj) [for short, an (•£,£)- 
Brownian motion]. The expression above for the covariance is obtained by 
calculating the covariance matrix of VF n (l) upon formally replacing the 
fluid-level quantities ^(s) in (26) by the quantities ip*j from the fluid model, 
and finally taking limits as n — ► oo. 

Next, equation (29) imposes a constraint on ty™ in the form of a lower 
bound at zero. It is also not hard to see that an upper bound on follows 
from (5), (14) and (18). However, such a bound is of the order of re 1 / 2 , and 
as n grows without bound, it becomes irrelevant. It therefore makes sense 
that, in the proposed diffusion model, each integral in the last term of (27) 
is replaced by a process r\ c that is required to have increasing sample paths, 
and no additional limitation. 

We thus obtain a diffusion model involving processes X(t), Y(t), Z(t) 
taking values in R^_, and, respectively, as well as M + -valued processes 
r?c, cGC. Equations (30)-(32) below, that describe the diffusion model, are 
analogous to equations (27)-(29), respectively: 

(30) X(t) = x + W(t) + f H(X(s) - Y(s), -Z(s))ds + V m c n c {t) 

Jo cec 

Yi(t)>0, ieT, Zj(t)>0, jej, 

(31) 

£ y i(*)=E*i(*) + E^(*)> t>o. 

iei iel jej 

(32) For every c EC, rj c is nondecreasing and r/ c (0) > 0. 

We will consider Y, Z and {rj c } as control processes, and regard (31), (32) as 
constraints that they must satisfy. To define precisely what controls will be 
regarded as admissible, recall that W is an (£, S)-Brownian motion defined 
on (0,JF, P). Let (Tt) be a right-continuous filtration of sub-sigma-fields of 
J- such that (W(t) — £t,J-t-t> 0) is a martingale. Let a deterministic initial 
condition xgl be given. A triplet (Y, Z, re) is said to be an admissible 
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control and X a corresponding controlled process if Y, Z and r\ are processes 
with cadlag sample paths, Y and Z are (J r t)-progressively measurable, n c is 
(.fi) -adapted, and (30)-(32) hold for all t € [0,oo), P-a.s. 

A principal hypothesis of all the results below is what we refer to as the 
null controllability condition: 

(33) There exists c € C such that e • m c < 0. 

Note that the cone C referred to in the Introduction, in which the increments 
of the /-dimensional process J2c m cVc(t) of (30) take values, is simply the 
cone generated by {m c :c€C}. Also, condition (1) of the Introduction is 
given explicitly by (33). The following result is proved in Section 3. 

Theorem 2.2. Assume (33) holds. Then there exists an admissible con- 
trol (Y,Z,rj) under which e ■ X (t) < and Y(t) = on [0,oo), P-a.s. 

The main results of this paper establish the validity of statements about 
the queueing model, which are analogous to Theorem 2.2 in an asymptotic 
sense. The first is concerned with preemptive scheduling (see Section 3 for 
a proof). 

Theorem 2.3. Assume (33) holds. Then there exist P-SCPs under 
which, for every e and T satisfying < e < T < oo, 

lim P(Y n (t) = for all t € [e,T]) = 1. 

The treatment of nonpreemptive scheduling is more involved. In order to 
keep the notation simple, we have focused in this paper on the most simple 
case where one can expect a null-controllability result: The case I = J = 2. 
Clearly, in this case there is at most one simple cycle. It is expected that the 
general case can be treated with similar ideas. The result below is proved in 
Section 4. 

Theorem 2.4. Assume I = J = 2 and let (33) hold. Then there exist 
N-SCPs under which, for every e and T satisfying < e < T < oo, 

lim P(Y n (t) = for all te[e,T]) = l. 

n — »oo 

Remark. As can be seen in Sections 3 and 4, Theorems 2.3 and 2.4 
hold with e = in the case that the initial condition x satisfies e • x < — 1 
(or even e • x < 0, with an obvious modification of the proofs). 
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2.4. Discussion. Let us consider some numerical examples. Consider first 
a system with structure as depicted in Figure 1(a). Assume Vj = 1 for j = 
1, 2, 3, and 

3 10 1 
1 4 2 



A = [a > M = M : 



(where in this subsection we abuse notation and label j by 1, . . . , J). One 
checks that the heavy traffic condition holds, and 

1 0.5 0\ f-7\ ( 9 

-2 



r= U 0.5 lj' mi = l 3 j' m2 



Above, mi and mi are the directions of control corresponding to the two 
nonbasic activities (2,1) and (1,3). In fact, the graph that appears in Fig- 
ure 1(b) precisely describes Q and Qb in the current example. Clearly, the 
null-controllability condition holds, since e • mi < 0. The simple cycles and 
directions of control are depicted in Figure 2(a) and (b). To demonstrate 
the geometric aspect, we refer to Figure 2(c), where the null domain X_ 
is shown along with a vector field defined on its boundary assuming the 
constant value m±. Under appropriate assumptions, one can construct a dif- 
fusion in R 2 with a boundary term according to this vector field, that will 
be constrained to X_. In contrast, ?ti,2 cannot be used to constrain the dif- 
fusion to the same domain. In general, the diffusion can be constrained to 
X_, provided that at least one of the vectors m c satisfies e • m c < 0. This is 
the source for condition (33). 

We next consider examples with two classes and two stations: 

_ /8 10\ /13 
^ = ' i= 3 6 ' A= 3 




Fig. 2. (sl) A graph as in Figure 1(b) with two simple cycles. Dashed lines represent 
nonbasic activities, (b) A possible set of directions of control corresponding to the two 
cycles, (c) The direction mi may be used to constrain the diffusion model to the null 
domain X_ . 
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(34) 



(35) 



r = (o as)' m= { 3 

2 I)' A= ( ? 2 5 



r = (o as)' m= ( 2 3 

Above, we have presented both the data and the solution to the linear pro- 
gram in each case. In both cases the heavy traffic condition holds and the 
activities are as depicted in Figure 3(a). Note that in these examples there 
is a single cycle. The null controllability condition does not hold in the first 
example, and it does hold in the second. In fact, one can write the null con- 
trollability condition (33) for the above examples in a straightforward way 
as 

(36) Atll+//22<Aii2+M21. 

It is instructive to note that there are values of \i for which (36) does not 
express the null controllability condition (33). Consider the following exam- 
ple: 

» = t i =(l A)' A =(n 5 5 

<•=(! »=(-* 

Note that the values of both sides of the inequality (36) are the same as 
in example (34). Still, as can be verified by calculating e • m, the null con- 
trollability condition, that did not hold in example (34), does hold in this 
example. The reason is that the structure of the graph has changed and it 
now corresponds to Figure 3(b). In particular, the direction of the cycle is 
reversed, and condition (33) is equivalent to (36) with a reversed inequality. 
We can see that the null controllability condition depends on the values of 
fj,, as well as the direction of the cycle, which in turn is determined by the 
fluid model parameters A and Ji. 

We now come back to the point referred to in the Introduction regarding 
the unusual heavy traffic behavior. A particular consequence of our main 
results can be stated as follows. One can find policies (preemptive and non- 
preemptive) under which, for every t > 0, 

(37) lim P(e • Y n (t) = 0) = 1. 



What we referred to as unusual is that a critically loaded system shows a 
behavior that is typical to underloaded systems: the possibility to maintain 
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Fig. 3. Two examples with cycles in opposite directions (dashed lines represent nonbasic 
activities). 

empty queues with probability approaching one. We would like to make 
precise the statement that the system under study is critically loaded. One 
aspect of this is that the underlying fluid model is critically loaded, in the 
sense that the linear program (10) is solved with p* = 1, by assumption. 
More significant, however, is a statement that can be made regarding the 
probabilistic model. Namely, one can show that in a probabilistic model 
associated with an overloaded fluid model, queues inevitably build up. 

To this end, let A£, jgl be constants satisfying A^ > Aj for all i and 
A'j > Aj for at least one i £ X. The parameters (A, p) lead to a fluid model 
that is critically loaded. In the same sense, the pair (A',/2) correspond to 
an overloaded fluid model. We consider a sequence of processes Aq L defined 
analogously to A n , but with X' n replacing the parameters A n , where A m is a 
sequence satisfying n~ l \' n — > A' [compare with (8)]. Let Xq L stand for the 
processes X n obtained by replacing throughout our probabilistic model A n 
by Aq L . Define analogously all other processes involved, for example, Yql 
in place of Y n . As the following result shows, the model thus obtained is 
indeed overloaded in the sense that queues (in fact, large queues) necessarily 
build up. The result shows a sharp contrast with (37). 

Proposition 2.1. There exist constants C\ and C2 > depending only 
on (A, A',/i) (and not depending on n or t) such that, under any policy, for 
every t, 

(38) lim Pie ■ Yq L (t) > (-Ci + C 2 t)n) = 1. 

n — >oo 

A proof is sketched in the Appendix. 

3. Diffusion model and queueing model in the preemptive case. In this 
section we prove Theorems 2.2 and 2.3. 

Proof of Theorem 2.2. Let c be such that e • m Co < 0. Consider the 
domain 

D a = {£ G R 1 : e • f < -a} 
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for some fixed a > 0. Fix some jo S J . We will construct a control with the 
following properties: 

Y(t)=0, Z j (t)=0, for all jo, 

(39) 

rj c (t)=0 for all c , t>0. 
The process Zj will satisfy 

(40) Zj (t) = -e-X(t), *>0. 

As a result, (31) will be satisfied. Finally, the process rj co will serve as a 
constraining term of a reflected diffusion on the domain D a with reflection 
field identically equal to m co on the boundary dD a . We therefore consider 
equation (30) in the special form 

(41) X (t) = x + W(t)+ /"* H(X(s))ds + m CoVco (t), 

Jo 

where H(£) = H(£, (e • £) e jo)- Note that H is linear. Consider first the case 
x E D a . The result of [10] regarding existence of strong solutions to stochas- 
tic differential equations with oblique reflection, stated for a bounded do- 
main, implies, using a standard localization argument, the existence of a 
pair (X, i] co ) with the following properties. The process X is progressively 
measurable, rj co is continuous nondecreasing, adapted, with values in R + , 
equation (41) holds for all t>0, a.s. and X(t) E D a , t > 0, a.s. In particu- 
lar, we have constructed a process X with the property e • X(t) < —a, t>0, 
a.s. Letting now Y, Z and 7/ c , c ^ cq be defined via (39) and (40), we have 
constructed a triplet (Y, Z, rj) that is an admissible control, and have shown 
that X and Y satisfy the conclusion of the theorem. 

In the case where e • x > —a, clearly, x + f3m co € D a for (3 large. Fix any (5 
as above, and set y = x + 0m co € D a . Denote the control and the controlled 
process corresponding to starting from y as (Y,Z,rj) and, respectively, X. 
Now set Y = Y, Z = Z, rj co (t) = (3 + ?j co (t) and X(t) = X(t). Then X(0) =y, 
and clearly, (41) still holds for all t > 0. As a result, (Y, Z, rj) is an admissible 
control and the conclusion of the theorem holds. □ 

We remark that in the proof above we can simply take a = 0. Our re- 
sults regarding asymptotic null controllability will be inspired by a similar 
construction, but a will be taken to be positive. 

Recall that, by definition, a P-SCP is a map that determines *$> n (t) for a 
given value of X n (t) in such a way that (2)-(6) hold. The following lemma 
shows that, under suitable conditions, we can determine the value of ^ n by 
first selecting values for Y n (t), Z n (t) and cE C}. 
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Lemma 3.1. There is a constant a>o > such that the following holds for 
all n large. Suppose that, for some t, the following relations hold: X n (t) G 
\\X n (t) -nx*\\ <a n, Y n (t) G Z+, Z n (t)eZ(, 

e • Y n {t) + e -N n = e- X n {t) + e • Z n {t), 
[e -Y n (t)]y [e ■ Z n (t)} < a In + 1, 

6Z+, cGC, and 

*c (*) ^ a o n > c G C. 

(42) = Gij(X n (t) — Y n (t), N n — Z n {t)) - J2 s(c,i,j)* n c (t). 

cec (ij)ec 

Then the quantities 

{x?(t),iel}, {Y?(t),iel}, {z?(t),jej}, {%(t),(i,j)elxj} 

satisfy (2) and (4)-(6). 

The proof of this lemma appears in the Appendix. 

Proof of Theorem 2.3. We begin by describing the policy. By Def- 
inition 2.1, we need to describe a map that determines *$> n (t) for a given 
value of X n (t). Given the one-to-one relations (16) and (18), this is equiva- 
lent to describing ^ n (t) for a given value of X n {t). Let ao be the constant 
from Lemma 3.1. If ||X n (t)|| > aon 1 / 2 , we assume that ^> n (t) is determined 
as the image of X n (t) under some fixed map in a way that is consistent 
with Definition 2.1 but otherwise arbitrary (there will be no need to describe 
f n more precisely). Focusing below on the case ||X n (i)|| < ao^. 1 / 2 , we fix a 
sequence K n , n G N, so that n l l 2 K n G Z + for all n and 

(43) K n — > oo, n~ 1//2 K n —tO asn^oo. 
Denote 

£>! = {£ GR J :e-£<-l}. 

Fix throughout a simple cycle cq for which e • m CQ < 0. Also fix throughout 
io G I and jo G J . The proposed policy sets 



(44) 
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and *£(t) = for all c / c , t > 0. It also sets Y; n (t) = for all i ^ i , 
Zf(t) = for all jVio and 

Y^t) = (e-X n (t)-e-N n ) + , 

(45) 

Z"(t) = (e-X n (t)-e-iV n )", t>0. 

By (43) and (44), *^,(t) < a n for all n large. By Lemma 3.1, are well 

defined given X n (t), and (2), (4)-(6) hold. This completes the description 
of the P-SCP. Clearly, this description, along with equation (3), uniquely 
defines the processes \E' ri and X n given the initial conditions and primitive 
processes. Although the description applies for any initial condition X°' n , 
the treatment with be slightly different for X°' n in D\ and outside. 
In what follows let < e < T < oo be fixed. Let 

(46) f n = inf{t:||X n (i)|| >a n 1/2 }. 
Let also 

H n (0 = H n (£ - (e • e - e • N n )+e io , -(e • £ - e ■ N n )~e jo ). 
For t < f n , one has by (27) 

(47) X n (t) = X°' n + W n (t) + J* H n {X n {s)) ds + J* *^(s) ds. 
Note that F n satisfy 

(48) \\H n {i)\\ < C H (U\\ + IliVl), ^M^GM, 

where Cj? is a constant independent of n. Denote = — e • m" . Note that, 
by assumption, C™ — > — e • m Co > as n — > oo. It is assumed in what follows 
that n is so large that 

(49) C n e > \e-m Co \/2=:2C . 



Denote X e n = e • X n and, similarly, W e n = e • W n , X»> n = e ■ X^ n and H, 
e- H n . Then by (44) and (47), we have 



X?(t) = X° e > n + W?(t) + f H?(X n (s)) ds 
(50) , J ° 



The rest of our argument is divided into four steps. 
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Step 1. We first show that there exists a deterministic constant C\ in- 
dependent of x, n and K n such that 

(51) ||X n ||^<Ci(l + ||l ' n || + ||W n ||^) on the event f n > T. 
To this end, denote A n = [4||e||(||X - n || + ||VF n ||^)] V 1 and let 

rf = inf{t E [0, T] : Xf (i) < -A n - 1}. 

Note that rf > 0. Since by assumption f n > T, (47) and (50) are valid for 
t < T. By (48) and (50), noting that X™(t) > -A n - 1 for t < rf , we have 

KnC? f Q ljt» W >-i < A n + 1 + /3„ + ||e||||X°' n || + ||e||||W re ||^ 

+ ||e||C H /V"(s)||^, i < rf A T, 
Jo 

where we denote /?„ = ||e||C j f/T||iV ri ||. Note that f3 n -> 0. Hence, by (47), 
\\X n (t)\\ <C 2 (A n + l) + C 2 /"*||X n (s)||efe, i<rf AT, 

JO 

where C2 does not depend on X 0,n , W n , K n or n. By Gronwall's lemma, 

(52) \\X n \\*n AT <C 2 {A n + l)e c * T . 

Since on the event rf > T the property (51) follows from (52), we consider 
in what follows only the case rf < T. If X%(t) < -1 for all t E [0,rf], let 
rf = 0; otherwise let 

rf = sup{tG[0,rf]:lf(t)>-l}. 
Note that Xf(rf ) > -1 - ||e|| ||X°< n ||. Hence, by (50), 
-A n + ||e||||l°' n || > lf(rf) -X n e (rf) 

= W?(t?)-W?(t2)+ r H2(X n (s))ds. 

It" 

As a result, 



A n / 2 < 



H™{X n {s))ds 



<\\e\\C H T?\\X n \\* T n+f3 n . 



This and (52) show A n /2 - j3 n < \\e\\C H T?C 2 (A n + l)e° 2T . Since A n > 1, 
A n /{A n + 1) > 1/2. As a result, there exists a deterministic constant 5 > 
not depending on X°' n , W n ,K n ,n such that, provided that n is large enough, 

(53) rf > 5 on the event f n > T. 
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Combining (52) and (53), we have that \\X n \\* s < C 3 (l + ||X°' n || + HW - "^), 
where C3 does not depend on X°' n ,W n ,K n ,n. In a similar fashion we have, 
for £ = 1, 2, ... , 

iixi* 5 <c 3 (i+iixi^_ 1)5 + iiinr). 

Assuming without loss that C3 > 1, the above implies 

\\X n \\* T < (2C 3 ) T/S+1 {1 + ||X°' n || + \\W n \\^) 

and proves (51). 

Step 2. Recall that W denotes an (£, £)-Brownian motion. We next show 
that 

(54) lim P(f n < T) = 

n— >oo 

and 

(55) W n ^W on [0,T]. 

Let Ai, i £l, and Sij, € T x J be mutually independent Brownian 
motions with mean zero and variances given by EAf(l) = XiC^j^ ESfj(l) = 
Hij. By Theorem 14.6 of [4], 

(56) (i n , S n ) (A, S) locally on compacts. 

By (5) and (6), tf£(t) < for all i,j and i. Thus, by (8) and (26), \\W n \\* T < 
||i n ||r + l|5' n |lc'4T+ \\ £n \\ T for a suitable constant C 4 . Hence, n _1 /2||W™||* 
converges to zero in probability. By (51), n" 1 / 2 ^ 

II^Af™ converges to zero 

in probability. Using the definition (46), this establishes (54). In turn, this 
shows that ra _1//2 ||X n ||y converges to zero in probability. By (45), so do 
n -i/2||fn||* and n -i/2||£n||*_ Note that G(x*,v) = ip*, as follows from (11) 
and (24). Using (42), linearity of the map G, (43) and (16)-(18), we thus 
obtain, for a suitable constant C5, 

" V>lr < n~ l \\G{X n - nx* - Y n ,N n -nv- Z n )||^ + rT x l 2 K n 

(57) < Cm~ 1/2 {\\X n \\* T + ||f n ||^ + ||iV n || + ||Z n ||^) + n~ l l 2 K n 
^ in probability. 

Combining (26), (56) and (57), the claim (55) follows from the lemma on 
page 151 of [4] regarding random change of time. 
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Step 3. Recall (19). We prove the theorem in the case x £ D\. In this 
case, for all n large, X°' n 6 D\. Denote 

r n = mi{se[0,T]:X-(s)>-l/2}. 

Denote also O n = {f " > T}. By (54), lim n P(n n ) = 1. If we show 

(58) lim P({r n < T} n ft n ) = 0, 

n — >oo 

it would follow that P(X£(t) < -1/2 for all i e [0,T]) -> 1 and, in turn, by 
(17) and (45), that P{Y n (t) = for all t G [0,T]) ->• 1, as n -> oo. Hence, in 
order to prove the theorem, it suffices to prove (58). 

To this end, note that the jumps of the process W n are bounded by n -1 / 2 
and write 

P({r n < T} n n n ) < P({there exist < s < t < T such that 

(59) -1 < X™(6) < -1/2 for all 9 G [s,t], 

K(s) < -7/8 and X?(t) > -5/8} D O n ). 

Under the event indicated immediately above, on the (random) interval [s,t], 
X™>-1, and thus, by (50), 

\<x^{t)-x:( S ) 

(60) 

= f H2(X n (9)) d9 + W2{t) - W?{s) - C™K n {t - s). 

J s 

Moreover, by (48) and (51), 

^H^{X n {9))d9<C H \\e\\C l {t-s)[r + \\W n \\* T \+p n , 

where r = 1 + sup n ||X°' n ||. Since K n — > oo, it follows that there is a deter- 
ministic tlq such that, for all n > uq, 

A n (s,t) : = W?(t) - W?(s) + C H ||e||Ci (t - s)\\W n \\* T + (i n 
>\ + [C?K n -C H \\4Cir]{t-s) 

(61) 

>\ + C K n (t-s) 



> 



C K}J\ t-s>K n 1/2 , 
1/4, t-s<K~ 1/2 , 



where Cq is as in (49), and on the first inequality we used the fact that 
C# ||e||Cir does not depend on n. Combining (59) and (61), we see that 
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there are constants Cq, Cj > not depending on n and K n such that 

p({t tl <T}nn n ) 

< P(there exist s<t<T such that A n (s,t) > CoK^ 2 ) 

+ P(there exist s<t<T such that t-s< K~ 1/2 , A n (s,t) > 1/4) 

<p{\\w n r T >c % Ki' 2 ) 

+ P(w T (W n ,K~ 1 / 2 )> 1/8 -f3 n ) 
+ P(C^||e||CiK n " 1 / 2 ||^ n ||^>l/8) 

< 2P{\\W n \\* T > C 7 K^ 2 ) + P(w T (W n , K' 1 / 2 ) > 1/8 - A,), 

where WT(f,S) denotes the modulus of continuity of a function / on [0, T]. 
Since by (55) W n are tight, and the weak limit process has continuous sample 
paths, (58) follows (see e.g., Theorem 16.8 of [4]) and, hence, the result. 

Step 4. Finally, we prove the theorem in the case x € D\. Let T3 1 = 
inf{t € [0,T] :X2 < Th en 

P({r 3 n > e} n n n ) < P({X2(6) > -1 for all 6 G [0, e]} n n n ). 

An argument similar to step 3 (only simpler) shows that, under the event 
indicated on the r.h.s. of the above display, 

A n (0,e) > -1 - r + \C n e K n - C H \\e\\C x r\e, 

and, in turn, that P(t^ > e) converges to zero as n — > 00. We can now 
review the argument of step 3, replacing X 0,n by X n (r^) and r n by T4 1 := 
inf{t e [r^,T] :X^{t) > -1/2} (where rl 1 = 00 on the event 7f > T) so as to 
show that P(t2 < T) — > as n — ► 00. The only issue that is different now is 
that the "initial condition" X n (T^) is random and cannot be bounded by 
a deterministic constant r. Instead, let us define the random variable r n := 
1 + ||X n (r 3 n )|| and let ft? := {r n < C (C H HeKd)" 1 ^} n {r 3 n < T}. Coming 
back to (61) with r n in place of r, it is clear that the second inequality will 
hold on Q™, and so the remainder of the argument of step 3 is valid once Q n 
is replaced by Q n flfi". Since P(t^ < T) converges to one, the relations (51), 
(54) and (55) imply that the random variables r n are tight and, therefore, 
P(fli n fi n ) — > 1 as n — > 00. This establishes the theorem. □ 

4. The nonpreemptive case. In this section we treat the nonpreemptive 
case and prove Theorem 2.4. Recall that we only consider the case I = J = 2. 
Thus, 1= {1,2} and J = {3,4}. The heavy traffic and complete resource 
pooling conditions, which are in force, imply that the graph of basic activities 
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Qb is a tree with vertex set XU J = {1,2, 3, 4} [cf. (12)]. It follows that there 
are exactly 3 activities that are basic. Had we not had a fourth activity, 
the graph Q would not contain a cycle and it would not be possible to 
fulfill the null controllability condition (33). Thus, the hypotheses of the 
theorem require that there be a fourth, nonbasic activity. We have labeled 
the classes as "1" and "2." Although J = {3,4}, it will be more natural 
in the discussion that follows, and throughout the section, to refer to the 
stations as "station 1" and "station 2," and with an abuse of notation, regard 
the index set for the stations as {1,2}. Accordingly, we have four activities, 
i,j € {1,2}, and without loss of generality, we let (2,1) be the only 
nonbasic activity [see Figure 3(a)]. As a result, the direction of the only 
simple cycle, which we denote as c, is the following: class 2 — ► station 1 — > 
class 1 — > station 2 — > class 2. By (21), we get m™ = (/i^ — /i™ 2 , —^21 + ^22) 
and m c = (/xn — ^12,-^21 + ^22)- In what follows we will write m n for 
and m for m c . We also let C m = —e ■ m n and C m = —e ■ m. Note that 
condition (33) can be written as C m = — e • m > 0. 

We now specify the N-SCP for which the conclusions of Theorem 2.4 will 
be shown. According to Definition 2.2, we must specify the initial arrange- 
ment and how routing is determined upon each arrival. To this end, we need 
some notation. Note that by (23) and (25) there exists a constant C' H such 
that 

(62) \\H n (a,b)\\ < ^(\\a\\ + ||6||), (a,b) eD G ,n£ N. 

Let 

2 + 16C^ 1 A log2 log8 

63 K = n ' 6= a 11 11 A ~~nT> 7 = —^- 

C m &K\\m\\ C H 

The initial arrangement and the routing rules will depend on X 0,n and, in 
particular, on whether e • X°' n < — 1 or not. First consider the case where 
e • X°' n < -1. 

Initial arrangement. Recall that X n (0) = X 0,n is given and we have to 
specify ^ n (0). We set 



(64) 


*5i(o) = 




(65) 


*ii(0) = 


|"(JVT - A^ 2 ™ + X°' n + A 2 °' n )/2] - ^(0), 


(66) 


*? 3 (0) = 


L(A- 2 n - TVf + X°' n - A 2 °' n )/2J + ^(0), 


(67) 


^22(0) = 




Using 


(4) and (5), one verifies that 


(68) 


Y?(0) 


= y 2 "(o)=o, \z?(o)-z$(p)\<i. 
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Routing. The routing decisions can be based only on n, the value of 
H n right before the arrival, and some auxiliary randomness that we have 
denoted by R n . The way we use the randomness in the proposed policy is 
so as to split the customers of class 2 into two sub-classes, which we label 
as a and (3. Upon the fcth class-2 arrival, an independent biased coin is 
tossed according to which it is decided what sub-class the arrival belongs 
to. The bias of the coin is allowed to depend on k. We denote by a n (k) the 
probability that the A:th class-2 arrival is classified as a class-a customer. 
The value of a n {k) is determined as 

Dince we assume that X°' n < -1 and the difference between Zf (0) and Z%(0) 
is at most 1 , it follows that the initial arrangement is such that there are free 
servers in both stations. Below, we shall describe the routing policy only as 
long as there are free servers in both stations. The description of the policy 
at other times is not important and will be left completely open (in fact, one 
of the main ingredients of the proof will be to show that the event that there 
are no free servers in one of the stations some time before T has probability 
approaching zero as n — > oo). The routing rules are as follows: 

1. Class- 1 customers are routed to the station with more free servers. More 
precisely, if a class- 1 customer arrives at time t, it is instantaneously 
routed to station j, where 



J 



1, if z?(t-)>z2(t-), 

2, otherwise. 



2. Class-a customers are routed to station 1 upon arrival. 

3. Class-/3 customers are routed to station 2 upon arrival. 

Next, consider the case where the initial condition satisfies e • X°' n > — 1. 
Denote r n = e ■ X°' n — e ■ N n + [n 1 / 2 ] . For the initial arrangement, we let 
r n class- 1 customers be left in the queue, and let ^(O) be defined as in 
(64)-(67), except that we substitute X°' n — r n for X 0,n . As a result, in 
place of the left part of (68) we will have Ff(0) = r n , Y 2 n (0) = 0. The rout- 
ing is determined as follows. The r n class- 1 customers that are initially 
put in the queue are kept in the queue. Rules 1-3 above apply for all the 
other customers. Upon the first arrival after the time Tq when the num- 
ber of free servers in the system first exceeds r n + [n 1 / 2 ] , all the r n cus- 
tomers are moved into service: 7" into station 1 and into station 2. Here, 
7 f = |W(r?-) - Z? (rf-) +r n )/2], 7? = L(^PW-) - Z^-) + r n )/2\. 
Clearly, e • ^""(tq 1 ) < —1. Also, by the above choice for 7", one achieves that 
37* (t£) = and |Zf (rft) - Z%(t$)\ < 1, a situation similar to (68). From the 
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time Tq on, the routing rules 1-3 above apply for all the new arrivals. Note 
that once again we have ignored the scenario that there are no free servers in 
one of the stations at some time before T, for reasons as mentioned before. 

Here and for most of this section we shall assume that e ■ x < — 1 . Since 
X°' n — > x, we have 

(70) e-X°' n <-l 

for all large n and, as a result, only the first part of the definition of the 
policy will be relevant until, at the end of the section, we return to the 
treatment of the case e ■ x > — 1 . 

Without loss of generality, assume that T is a multiple of 5, and set k = 
T/5. Divide the time interval [0, T] as follows: Iq = {0}, I\ = (0, 6], . . . , 1^ = 
((k - 1)5, k5] . Let h : [0, T]^lbe defined as 

h(t) = 2~ k for all t € Ik, k = 0, 1, . . . , k. 

Define 

(71) r n = inf|t>0:X e n (t)>-^| AT. 
Let b! = ±2~ k = ^h(T). Let 

(72) a n = inf{t > : Z?(t) A Z£(t) < Ah'} 

and Q n = r n A o n . By (70) and since ^f(0) and ^(0) are nearly equal 
[cf. (68)], it is useful to note that 

(73) Z?(0) A l£(0) > 1/4. 

Also, it is clear that there are idle servers in both stations up to time a n . 
Denote 

D(a,p) = {(£R / : ||f || < a, e • f < -p}, a > 0, p > 0. 
Let K n = n 1/8 and 

^ = D( e ^Er n ,2- fc ). 

For k = 0,1, . . . ,k, denote 

(74) ni = {X n (t) eDl, te I v , for all k' < k} n {C > 

Proposition 4.1. Let the assumptions of Theorem 2.4 hold and assume 
also that e ■ x < —1. Then for k = 0, 1, . . . , k, P(Q2) — » 1. 
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The above result implies that P(a n < T) — > 0. Since by construction of 
the policy, Y n (t) = for t <o~ n , this establishes Theorem 2.4 in this case. 

In preparation for the proof of Proposition 4.1 we need some notation 
and preliminary results. Recall that we have denoted the only cycle by c, 
and that, according to our notation, = *2i(*) anci (*) = *2i(*)- Let 

A™(i) [Ag(f)] denote the number of class-a (resp., class-/?) customers that 
have arrived up to time t. Let 

(75) A n a (t) = n- l ' 2 A n a {t) - K n [\^ 21 + 7 )^( a ) <fe, 

JO 

where 

(76) tfj (t) = ne 11 , t>0. 

A sequence of processes is said to be C-tight if it is tight and every sub- 
sequential limit has continuous sample paths with probability one. The fol- 
lowing three lemmas will be proved later in this section. 

Lemma 4.1. Under the assumptions of Proposition 4.1, one has 

(77) \A™\t ^0 in probability, 

(78) \\X n - x*\\* TAa n = n~ 1/2 \\X n \\* TAan ^ in probability, 

(79) V1ta<7" =n~ l/2 \\i> n \\* TAan ^0 in probability. 

In addition, the processes Af(- f\o n ), S^- (Jo' A<7 tyfjs)ds) are C-tight. In 
particular, one has {S^iHq ^ , 2i( s ) ^ s )Ita<t™ — > in probability. 

Lemma 4.2. Under the assumptions of Proposition 4.1, — 
-KhV'oItao-" — > in probability, where tpo is as in (76). 

Lemma 4.3. Under the assumptions of Proposition 4.1, lin^^oo 

T n ) =0. 

The following will be used several times in the proofs of both Proposition 
4.1 and the above lemmas. Recall that, for t < a n , Y n (t) = 0. Hence, we 
have from (27) 

(80) X n (t)=X°' n + W n (t) + f t [H n (s)+m n ^(s)]ds, t<a n , 

Jo 

where we have denoted 



(81) 



H n (s) = H n (X n (s),-Z n (s)). 
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Also, substituting zero for Y n in (28), using (14) and positivity of Z™, we 
have, for all n large, 

(82) \\Z n (t)\\ < \\X n {t)\\ +1, t<a n . 
Thus, by (25), (62), (81) and (82), for all n large, 

(83) \\H n (s)\\ V ||e • H n (s)\\ < C' H (1 + \\X n (s)\\), t < a n . 

Proof of Proposition 4.1. We argue by induction on k. As the in- 
duction base, consider k = 0. By (70) and since X 0,n converge, we have 
X n (Q) E Dq for all large n. By Lemma 4.3, it remains to show that r n > 
with probability approaching 1. This follows from (71) and the fact that the 
jumps of X n are bounded by n~ 1 / 2 . 

Next consider the induction step. Assuming that for a given k < k one 
has 

(84) re limPTO = l, 
we shall prove 

(85) taQP(ng +1 ) = i. 

In view of Lemma 4.3, it suffices to show 

(86) lim P(r n <(k + 1)5) = 0, 

n — >oc 

and 

(87) Jim P(\\X n \\* s[k+1)Aan < e^ k+1 ~>K n ) = 1. 
To this end, note that from (80) we have 

(88) X n e {t) = X° e ' n + W?(t) + f[H n e (s) - C^ n c (s)\ ds, t < a n . 

Jo 

By (63), one verifies that the constants k, 5 and 7 satisfy 

(89) e c 'x 5 < 2, 1 + C' H e~< 5 < \C m K, 

(90) 4(l + |H|K5e 7<5 )<e 7<5 . 
Denote 

njf = fi£ n {a n > T n } n {r n <(k + i)S}. 

Let h= |2 _fc . On £Vj: (hence, on fl^) the following must hold: 

(91) 11^(^)11 < e^ Sk K n , X r e \k5) < -8h. 
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In addition, on 



there exist k5 < s <t < (k + 1)5 A a n such that 
K{s) < -7h, X™(t) > -5h. 
Hence, in view of (88), the following must hold on Ct^: 
2h<X?{t)-X n e {s) 

(92) 

= W?(t)-W?(s)+ f [H2(u)-CX(u)]du. 

J s 

Denote a n = \fy - K n ^ Aan , b n = \\W n \\* TAan and d» v = W?(v) - W?{u). 
Let 

(93) &l = fi£ fl {C m a n < 2} n {2b n + 25||m||a n + T < K n }. 
By Lemmas 4.1-4.3 and (84), 

(94) ta?(nj) = l, 

On the event £l%, by (80) and (83), denoting 

A n = \\X n (k5)\\+2b n + 5\\m n \\\* n c \* TA(7n +T, 

we have 

\\X n (u)\\ < A n + rc^H^^OUdu', u<(Hl)iAa n . 

By Gronwall's inequality and the first part of (89), this shows that ||X n (?x)|| < 
2A n , for u as above. Using this along with (93), and assuming in what fol- 
lows that n is so large that \\m n \\ < 2\\m\\ and > C m /2, we have on the 
event 

\\X n {u)\\ < AK n e^ k {\ + ||m||/^e 7<5 ) < K n e^ 5{k+l \ 

(95) 

u<(k + l)5Aa n , 

where in the last inequality we used (90). Equations (94) and (95) establish 
(87). Next, combining (92) and (95), and using again (93), we have on the 
event ft£ n 

2h < dl t + (C' H + l)(t - s) + K n {t - s)e i&k [C H e^ - C m n/2]. 

Writing C = C' H + 1 and using the second part of (89), we conclude that, 
for all large n, on the event fi£ n £1%, there exist s and t such that kS < s < 
t < (k + 1)5 A o n and 

f ±K 1/2 t-s> K~ 1/2 
d n st >2h+{e l&k K n -C){t-s)>\^ n ' 1 " n ' 

I 2h, t-s<K n y \ 
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Hence, for all large n, 

P{n%) < P((n%) c ) + P(there exist < s < t < T A a n such that 2d% >t > K^ 2 ) 
+ P(there exist 0<s<t<TAa n 

such that t - s < K~ 1/2 and d™ t > 2h) 
< P((h n k ) c ) + P(Ab n > K]l 2 ) + P(w TAan (W n , k- 1 ' 2 ) > h). 

By (94), the tightness of b n and the C-tightness statement in Lemma 4.1, 
we see that -P(f2£) — > as n — > oo. By (84) and Lemma 4.3, this shows (86). 
□ 

Let a constant r > be given and let 9 n = inf{i G [0, 5] : X™{i) < —r}. Let 
e be as in Theorem 2.4. The following consequence of the above proof will 
be useful. 

Corollary 4.1. Under the assumptions of Proposition 4.1, 
lim n _ >oo P(0 n >e/2) =0. In addition, \\X n \\g n are tight. 

Proof. With the notation from (74) and (93), define Q n = {6 n > e/2} n 
By Lemmas 4.1—4.3 and Proposition 4.1, hm^—xx, P(fl r {) = 1 and, there- 
fore, it suffices to show that P(Q n ) — > 0. On Q n we have 

-r < X2(e) - X2(0) = W2(e) - W?(0) + / V e » - du, 

Jo 

and an argument along the lines of the proof of Proposition 4.1 proves the 
first claim of the result. 

Next we prove the tightness statement. Note that, by assumption, ||X n (0)|| < 
Co for some deterministic constant Co independent on n. Let ? n = (2r + 
||e||C )e-^Er- 1 and Q n = {6 n > ? n } n Aj. On Q n we have (just as in the 
proof of Proposition 4.1) 

-r < K(n = $2(0) + W?(, n ) + f [H:(u) - CX(u)] du 

Jo 

< X™(0) + W:^ n ) + (C' H + l) ? n - K n 
<W?(s n ) + {C H + l)<; n -2r. 

As a result, 

(96) lim P(9 n > q n ) = 0. 



QUEUES IN HEAVY TRAFFIC 33 

Let C = Co + 4r||m||K + 2C#r + 1. On Cl™, for n so large that ? n < 5, with 
the notation a n = j^™ — K n ^o\^ Ao . n , we have 

||X n ||*„ < \\X n {0)\\+w T {W n ,<; n ) 

+ \\m n \\5a n + \\m n \\KK n e~<\ n + C H K n e~<\ n 

< w T (W n , ? n ) + 2||mpa n + C - 1. 

Since, by Lemmas 4.1 and 4.2, we have lim n _ >00 P(wT(W n , s n ) + 2\\m\\5a n > 
1) = 0, we conclude that lim n _^oo P(||X n ||*„ < C) = 1. In view of (96), this 
shows that ||X n ||g„ are tight. □ 

We turn to prove Lemmas 4.1-4.3. 

Proof of Lemma 4.1. We first prove (77). Let K% denote the indicator 
of the event that, in the nth system, the A:th class-2 arrival was classified 
as an arrival of class a. Then EK% = P(Ru = 1) = a n {k) given in (69). 

Since A™(t) represents the number of class-2 arrivals up to time t that were 

A n it) 

classified as class a, we can write A™(t) = J2k=i R k- By (75), we have 

Al(t) = n-V 2 A n a (t) - nV8 ML ( e 7* _ i), 

7 

where throughout this proof C = {n2i + 1)^/^2- Fix e > 0. Then A™(t) > e 
if and only if A™(t) > n l / 2 e + K™, where 

X? = n 8/8M( e 7t_l). 

7 

Therefore, 

(97) P [there exists t < T, such that A n a {t) > e] < p? +p%, 

where 

p? = Pfsup \A%(t) - X£t\ > n 3 / 4 ) ^ asn^oo 

\t<T J 



(98) 



2 = P \ sup \A%(t) - \%t\ < n 3/4 , and there exists t < T, 

\t<T 

A n (t) \ 
such that R k > nl,2e + K t 

k=l / 
/ P(n,t) \ 

< P there exists t < T, such that J2 R k ~ K t > ™ 1/2e , 
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where p(n,t) = [X%t + n 3 / 4 \. The convergence statement in (98) is due to 
the tightness of M± [cf. (56)]. A direct calculation based on (69) shows 

(3(n,t) 

£ E(R n k )<K? + C'nV s , 

k=l 

for all large n, where C is a deterministic constant not depending on n. 
Hence, 

(3(n,t) 

there exists t < T such that J2 ( R k ~ ER l) > ™ 1/2 £ ~ C'n 3/8 . 



p n 2<P 

Denoting m(l) = J2k=i( R k ~ ER k)j we can wr ite 



■J. 



P n 2 < P(H* 2X n T > < 16 g M 2A S T ) 2 ] 



e z n 

where in the last step we used Doob's inequality for the martingale m. 
In turn, E[m(l) 2 ] =X)L=i Q n(^)) and substituting 2A2T for I, one finds that 
P2 — > 0. As a result, the l.h.s. of (97) converges to zero. A similar calculation, 
which we omit, shows that the probability that there exists t <T such that 
^-a(i) < — £ also converges to zero. Since e is arbitrary, (77) follows. 
Exactly as in the proof of Theorem 2.3, we have 

(99) n~ 1/2 W n 0. 

Since X°' n converges, we have from (80), (81) and (83), for all large n, 
\\X n (t)\\<K n +\\W n (t)\\ 

(100) t 

+ C' H ( {l + \\X n {s)\\)ds + T\\m n \\\\^\\* t , t<a n . 
Jo 

Using the inequality 

(101) i> n c {t/\o n )<i> n c {Q) + n- l l 2 Al{T) 
and GronwalFs inequality, we have 

(102) ||X n (t)||<Ci(iif n + ||W n (t)||+n- 1 / 2 ^(r) + l) J t < a n AT, 
for an appropriate constant C\ not depending on n. It follows from (77) that 

(103) lim P(A™(T)>n 3 / 4 ) = 0. 

Combining (99), (102) and (103), we have (78). Using (4), (5), (11), (16)- 
(18) and the fact that Y n {t) = 0, we have, for all t < a n , 

Z^(t) + ^ l {t) + i> n c (t) = N[ l . 
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These equations along with (82) show that 

(104) ||tf"(i)|| <C 2 (l + \\X n (t)\\ + ^(t)), t<a n , 

for a constant C 2 not depending on n. Combining (101), (103) and (78), we 
have (79). The result (79) implies that the processes Act") are tight and 
that every subsequential limit has continuous sample paths with probability 
one. Using this and the time change lemma [4] along the lines of the proof 
of Theorem 2.3 above proves C-tightness as claimed. Finally, consider the 
last claim in the statement of the lemma. Since tp 2 i = 0) (J9) shows that 
I Itact 71 ^ in probability. Using this and (56), the claim follows using 
again the time change lemma. □ 

Proof of Lemma 4.2. By construction of the policy, B^it) = A™(t) 
for all t < a n . Thus, by (7), 

= *2! (0) + K (*) " ( jf *2! 

Denote 

(105) g(t) = {^i+i)Mt), t>o. 

Recalling that = n^ 1 / 2 ^ (t) = n" 1 / 2 ^^), using (75), one checks by 

direct calculation that 

= ^(0) + n 1 / 8 / (is - / ds + Wo*(<)> * < 

where 

TU "(t)=iS(t)-^ 1 (/ o < ^ 1 (s)d s ). 

Note that «/>o(i) = K + Jo @( s ) ^ s ~~ Jo ^o(s) <^s, and let Vo De the unique 
solution to 

(106) %(t) = k + f q{s) ds - fa f W{s) ds. 

Jo Jo 

Then, for t <a n , 

- n^Vo (t) = *c(°) - nl/§K 

- tfi A^W - (*)] cfa + Wff(f). 

Jo 

By (64), #£(0) - n x l*K -> 0. By Lemma 4.1, \W$\* T ^ n -> in probability. 
An application of GronwalFs inequality therefore shows that 

(107) - n 1/8 ^o Ita<t« ^0 in probability. 
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Also, by (13) and (106), it is easy to see that 5 n (t) < Ce n + C / * 5 n (s) ds for 
t <T, where 5 n (t) = \ipo(t) — ipo{t)\, C is a constant and n 1//8 e n —> 0. Hence, 
n 1 / 8 ^™!^ — > 0. Along with (107), this proves the lemma. □ 

Proof of Lemma 4.3. By (73), the initial values for both Zf and Z 2 
are greater than 8h' . We thus have {a n < r"} C $7™ U , where, for i = 1, 2, 

$1™ = | there exist ui < U2 < Q n such that 

sup Zf(u) < 7h',Z?( Ul ) > 6h',Z^(u 2 ) <5h'\. 



Step 1. We show that P(^i) — > 0. Let ui and «2 be as specified in the 
expression for f2™. Note that the size of the jumps of X n is bounded by 
n -1 / 2 . Hence, for u < r n , we have X^(u) < —32h' + ?i -1 / 2 . This, combined 
with (14) and (28), and the fact that = for u < a n imply that 

Zf(u) < Z' 2 (u) for u € [ni,U2]. According to our definition of the policy, all 
class-1 customers are routed to station 2 during the period [tii,ti2] an d only 
class-a customers are routed to station 1 during this period. As a result, 
B^(u 2 ) - B^{u x ) = and B^{u 2 ) - B^( Ul ) = A%(u 2 ) - A n a { Ul ). By (7), 
we have 



E 

i=l,2 



(v^iK) - = Al{u 2 ) - Al( Ul ) - D?( Ul ,u 2 ), 



where 
D](h,t 2 ):-- 



E 



=1,2 



da 



.7 



1,2. 



Note that D^{t\,t 2 ) represents the number of departures from station j 
during (ti,t 2 ]. Using (5) and (14), one has on Q± that 



(108) 

Denoting, for j 

W?(t) : 



tin 1 ' 2 < A n a {u 2 ) - A n a { Ul ) - D?( Ul ,u 2 ). 



1,2, 



: E s % 

i=l,2 



m{s)ds 



(109) S2(h,t 2 ) = W?(h 



ft. 



i=12 l Jt! 
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and 

AUh,t 2 ) = A n a (t 2 ) - AHh) + n 5 / 8 [ t2 g(s) ds, 

Jti 

where g is as in (105), one checks by direct calculation that (108) can be writ- 
ten as h' < Ai(ui,U2) — Si(ui,u 2 ). Note that g is bounded above. Moreover, 
Mil i s bounded below by a positive constant. Using (104) and denoting 
A = u 2 — ui we therefore have, for all large n, 

(110) ti < 2\A^\* T + w TA(7 n(W^ A) - n^A^A, 

where 

AJ = Ci - C 2 n-V2(i + + 

and Ci > and C2 are suitable constants. We conclude that P(n™) < 
P(n?,i) + P(n?, a ), where 

fi^ = {there exists A G (0,n" 1/4 ] such that (110) holds}, 

fi™ 2 = {there exists A G (n~ 1/4 ,T] such that (110) holds}. 

By Lemmas 4.1 and 4.2, A" — » Ci and \A™\j, ^ in probability, and Wf (• A 
<r n ) are C-tight. This shows that P(O^) -> 0. On n? )2 , if A? > 0, then 
n i/4 A n < £n mugt hold) where ^ = 2|ij|* + 2|Tyf|^ A(jn . Hence, P(^ 2 ) < 
P(A? < 0) + P(A? < n" 1 / 4 ^) -f 0. This shows that P(fif) -» 0. 

S'tep 2. We next show that P(rir>) — > 0. Letting u\ and 1*2 be as in the 
expression for an d arguing as before, one obtains that Z™{u) > Z 2 (u) 
for u G [1*1,1*2] and, consequently, that all class- 1 customers are routed to 
station 1 during this period. Analogously to (108), we find that on 

tin 1 ' 2 < A n p (u 2 ) - A% Ul ) - £>£(ui,U2). 

Using the inequality A r ^(u 2 ) — Ag(ui) < A 2 (1*2) — ^2 ( n i) an d some direct 
calculation, we deduce from the above that 

(HI) ti <A^( Ul ,u 2 )-S^u u u 2 ), 

where S 2 is as in (109) and 

if (h,t 2 ) = A n 2 (t 2 ) - A n 2 (t x ) + \ n 2 n^l\t 2 - h). 

Note that the r.h.s. of (111) contains the term j n nl ' 2 ( u 2 — ui), where 7„ = 
n _1 A2 — J2i=i 2 ^i^^Ph- We claim that j n is bounded above by a negative 
constant for all n large. Indeed, recall that £21 = and note that from (10) we 
have A 2 = jl 22 Q 2 - With (9) and (11), this shows A 2 = fJ- 22 ip 22 < Z)i=i,2 /^V^V 
The claim regarding -f n thus follows from (8). 
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Along the lines of step 1, we obtain, instead of (110), 
ti < W TA a" (i<j - A) - n^AJA, 

where A = u-2 — u\, 

A£ = C 3 - Cm- X, \\ + \\X n f Th(jn + l*c Itao»), 

and C3 > and C4 are constants. The rest of the argument for showing 
P(firj) ^ is similar to that in step 1 and is omitted. We conclude that 
p\a n < r n ) -► 0. □ 

Proof of Theorem 2.4. In view of Proposition 4.1, it only remains to 
treat the case where e-x > —1. We can split the sequence of systems into two 
subsequences according as e-X 0,n < — 1 or not, and since the result is already 
proved for the subsequence on which e • X°' n < —1, we will assume, without 
loss of generality, that e • X 0,n > — 1 for all n. As a result, the second part of 
the definition of the proposed policy applies. In the first part of the proof we 
used the fact that the system starts with no queues. In the current situation 
we argue that, even if the system starts with a queue, it is brought quickly 
to zero. Recall the notation Tq from definition of the policy. Note that, prior 
to time Tq, there are r n class- 1 customers that are kept in the queue and 
that, as far as all other customers are concerned, the system behaves exactly 
as in the first part of the definition. As a result, we can use Corollary 4.1 
with r = 1 + sup n n~ l / 2 r n < 00, and it follows that P(tq < e/2) converges to 
1. At time Tq the system is in a state very similar to that in which a system 
satisfying e • X 0,n < —1 is at time zero, in the sense that e • X u (tq) < — 1 and 
\Z"i(tq) — ^(tqOI < 1. A review of the proof of Proposition 4.1, replacing 
the initial values of all processes by their values at time Tq, shows that, with 
probability approaching 1, Y n (t) is kept zero for t G [e, T\. As in Section 3, 
the only remaining issue is that ^"(tq 1 ) must be shown to be tight. This is 
indeed the case by Corollary 4.1. □ 

APPENDIX A 

Lemma A.l. (i) All nodes lUj of Qb are connected through the edges 
of Qi). (ii) Every nonbasic activity (i,j) € £ n b belongs to a simple cycle. 

PROOF. For (i), consider a node of Q, i G I. By (10), ^7 PijCtj > and, 
therefore, there exists j 6 J such that > 0. This shows that (i,j) £ £b 
and thus, i must be a node of Qb- A similar argument holds for a node j € J 
observing that = 1 by the heavy traffic condition. Item (ii) follows 

since Qb is a tree. □ 
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Proof of Theorem 2.1 . We first show that the relations (3)-(6), when 
rescaled appropriately, imply the following: 

(112) *°'" + WT(t)-5>5 ds, iexjej, 



(113) 
(114) 



(115) li(i)>0, ^j(i) > 0, *™(i)>0, 
To this end, note that by (15), (16) and (19), 



ielj eJ,ceC,t>0. 



+ ri 



-1/2 



By (13), (17) and (18), the last term in the above display is equal to 



-£/4 

Using (10), (11) and (13), 



^ - E 



t. 



n 



1/2 



\i - E 



■ E 



Combined with (26), this establishes (112) above. Equations (113) and (114) 
and the first two inequalities in (115) directly follow from (4), (5), (6), (11), 
(14), (16), (17) and (18). For the last inequality in (115), recall that, for 
c G C, iff™ = iffy, where = <t~ 1 (c), and that, for all of this form, 

^• = o. 

Equations (28)-(29) follow from (113)-(115). As for (27), it suffices to 
prove the identity 



(116) 



*? j (t) = G ij {X n (t)-Y n (t),N< 



Z n {t))~ E s(c,i,j)^ c ( t ), 

c£C:(i,j)€c 
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for all (i, j) ES, t>0. Indeed, with (21), the above implies the identity 
" E = - E ^G i3 (X n {t) - Y n (t),N n - Z n {t)) 

cec 

which, along with (25) and (112), establishes (27). In order to show (116), 
define 

(117) T£(i) = s(c,i,j)^ c (t), (i,j)eS. 

ceC: (i,j)€c 

As follows from the uniqueness statement succeeding (22), in order to prove 
(116), it is enough to show that {T^} satisfy the system of equations (22) 

with a = X n (t) - Y n (t) and b = N n - Z n (t). To this end, note that if is 
a nonbasic activity, then there is exactly one simple cycle c for which is 
an edge and one has s(c,i,j) = —1 and, by definition of VP" = This 
shows that T^- = for £ £ n b- Next, note that, for a given c 6 C and a 
node i € I of c, there are exactly two edges of c of the form ■ By the con- 
struction of directions, for these two values of j, the signifiers s(c,i,j) must 

have opposite signs. As a result, J2cec^2j-. U,j)ec s ( c > h j)^" '(*) = 0- Changing 
the order of summation and using an analogous argument for summation 
over i, we obtain 

E E ■s(c,i,j)^(t) = o, jej. 

iel ceC: (i,i)6c 

Equations (113), (114) and (118) now show that {T™ } satisfy (22) with the 
values a and b mentioned above. This proves (116) and, in turn, the relation 
(27), and completes the proof of the theorem. □ 

Proof of Lemma 3.1. Let X n (t), Y n (t), Z n {t) and be given, 

satisfying all requirements in the statement of the lemma. Define {^fj(t)} ac- 
cording to (42). Equations (2), (4) and (5) follow directly from the definition 
of G and {s(c,i,j)}. As for (6), it remains only to prove that > for 

every € £b- Along the lines of Lemma 3 of [2], note that ip* = G(x*, u), 
as follows from (11). Using linearity of the map G on the domain Dq and 
the bound assumed on \T r ", 

*S(t)>G y (nx*,m/) 

+ Gij{X n (t) - nx* - Y n (t),N n -nv- Z n (t)) - C x a Q n, 



QUEUES IN HEAVY TRAFFIC 41 

where C\ is a constant independent of n, t and the choice of ao and, therefore, 

^ 3 (t)>(^-C iao )n 

- C 2 (\\X n {t) - nx* - Y n (t)\\ V \\N n -nu- Z n (t)\\), 

where Ci is a constant depending only on the map G. With the assumed 
bound on \\X n — nx*\\ V \\Y n \\ V \\Z n \\, using also (14), we obtain $??j(t) > 
(if)*j — C^ao)n — C^n 1 / 2 for appropriate constants C3, C4 independent of n, t 
and ao- Since tp^j > for all £ St,, it follows that ao > may be chosen 
so that, for all large n, V^-(i) > 0. □ 

Sketch of proof of Proposition 2.1. We simplify the notation by 
writing A n , A n , X n and, respectively, X n for Aq L , Aq L , Xq L , Xq L , and so 
on (there will be no confusion). Note that an analogous result to (56) holds 
and, in particular, for every t, 

(119) n^Aiit) -+t\i in probability, 
as n — > oo. Fix t>l. By (112), we have 

X? (t) = X?(0) + A?(t) - J2 fMj Jl ds + A?(t), 

where 

(120) = e [«~ i/2 ^-(jf m*) ds ) + k - ^i) jl m*) ds 

By (5) and (6), ^^(t) is bounded. An argument as in step 2 in the proof of 
Theorem 2.3 shows that the first term in (120) converges to zero in proba- 
bility. Since fifj — > fiij, so does the second term and, in turn, A™(i) — ► in 
probability. It can be shown that there exists a constant 5 > depending 
only on (A, A',/2), such that, for any sub-stochastic matrix (£r,), 



(121) max 



L 3 

Let n n = {for all i El, |Af(i)| + |i _1 ^f(i) - AJ| < 5/3}- By (119), we have 
P(n n ) -> 1. Define 

1 cn _ Q 



ij 



It can be shown by (5) and (8) that A - - £)j > A ■ - X)j ftij 6$ ~ <V 3 , f° r 

all n large and all i. Since £ is sub-stochastic, (121) implies that, for some i, 
we must have A- — J2j ftij > ^> hence, A- — /2jj > 2 J/3, for some i. 
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This can be used to show maxjX™(i) > St/3 on J7 n . Due to the relation 
+ Ej ®ij(t) = Xf(t), we have Y?(t) > Xf(t) - C, for all i, where C 
does not depend on n or t. Hence, max, YJ n (i) > St/3 — C. We conclude that 
on Q n , for all n large, Ei^ n (*) > [St/3 — C]n, and the result follows. □ 
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